The present invention relates to a method for providing a tutorial message to a driver of a vehicle for encouraging a desired driving behavior of the vehicle. The invention also relates to a corresponding tutorial system and to a computer program product.
The operation of a vehicle involves a large plurality of different tasks, including for example the more general issues relating to distance and lane keeping, awareness in general (e.g. when driving on a straight road or entering a roundabout), and more complicated traffic situations as well as more specifically in relation to safe and economic operation of the vehicle.
Modern vehicles are typically provided with different systems for simplifying these tasks, using for example electronic systems for optimized gear shifting, adaptive cruise control systems taking into account distance keeping, etc.
Additionally, much progress has been made in implementing educational systems in the vehicle for providing feedback to a driver of the vehicle for pointing towards an unsafe operational behavior of the vehicle. Such system may for example allow for the possibility to instruct the driver to take a break in case of the system identifying the driver to becoming drowsy, or for providing a feedback in case the driver fails to apply to regulations relating to the operation of the vehicle. For example, instant feedback may be provided to the driver in case he/she is distracted by a hand-held smartphone (e.g. looks away from the road for long periods of time).
Even though improvements have been made in correcting the driver when failing to be law abiding or not driving in a safe and economic manner, it would be desirable to provide further improvements for encouraging the driver to operate the vehicle according to a desired driving behavior. Specifically, it would be desirable to allow for the possibility of adjusting the driver feedback to in the most optimal way make the driver conform to the expectations placed on the driver in a specific driving context.
According to an aspect of the invention, the above is at least partly met by a method for providing a tutorial message to a driver in a vehicle using an in-vehicle tutorial system comprising a multimodal user interface, the method comprising: assigning the tutorial system with an area of improvement, comprising at least one tutorial task, identifying a driving context for which the tutorial task is to be performed, based on the tutorial task and the driving context, selecting a tutorial message encouraging a desired driving behavior, and providing the selected tutorial message to the driver using the multimodal user interface.
An assigned area of improvement may be general and describe a whole domain, such as for example “efficient driving” or “safe driving”. In other contexts or embodiments, an area of improvement may be more specific and describe a lower level category that could be a sub-part of the mentioned domains, for example “economic driving/fuel consumption”, “wear and tear”, “inattention” and “risk behavior”. The categories themselves may comprise specific tutorial tasks or situations that may be a drivers day to day driving tasks, such as for example how to shift gears in a fuel efficient manner in a steep hill, how to handle the vehicle in a difficult crossing or safe headway distance keeping. In some contexts or embodiments it may be possible to assign one tutorial task or a plurality of tasks as the area of improvement.
The invention is based on the understanding that an efficient and pedagogic manner of improving a driver behavior with respect to various aspects of driving is to encourage a desirable behavior of a driver in a vehicle in response to various driving events. The area of improvement is assigned to a tutorial system which then presents tutorial messages to the driver when a driving context where tutoring is required is identified. Thereby, the tutorial message can be based not only on the specific task, but also on the conditions under which the task is to be performed, such as for example external-, vehicle- or driver related conditions.
A driving context may comprise an operational context of the vehicle and a driver context. The operational context of the vehicle may then include environmental conditions such as rain, snow, dark etc. Operational context may also include vehicle speed, heavy traffic, lane change, queue ahead and similar vehicle and traffic related conditions. The driver context on the other hand relates to the state of the driver and may refer to parameters such as driver drowsiness, workload, distraction level, intent, impairment etc.
The driver is provided with a vehicle-integrated driver tutorial system which uses information from basic vehicle systems (e.g. engine system, climate system, tachograph, etc.) active safety systems, systems that capture critical driving events (e.g. incidents, near-crashes, crashes), systems that capture targeted tutorial events (e.g. how to handle a particular vehicle-trailer combination optimally on certain road segments), and systems that are capable of transferring information (e.g. CAN data, active safety system data, extra sensor data such as smartphone accelerometer or GPS data) and video (e.g. video captured from cameras integrated in the vehicle, from a smartphone, or from extra cameras) using telematics (e.g. in-vehicle telematics systems, smartphones) to and from an off-vehicle analysis center (e.g. a training center providing on-line chat service, recorded messages, manual or automatic analysis).
The tutorial message may preferably be provided in the form of an audio, voice, visual, or haptic output, or combinations thereof, from the multimodal user interface of the tutorial system. A multi-modal interface should be understood as an interface capable of providing an output to the driver through at least two of the modes audio, voice, visual, or haptic output. The form of the delivery of the tutorial message naturally depends on a range of parameters such as type of event, environmental circumstances, and driver condition. For example, a tutorial message encouraging the driver to look to the left in an intersection may combine a voice command with a visual indication. Furthermore the visual indication may either be based on the desired behavior, e.g. a visual indication in the left part of the windscreen. Alternatively, or in combination, the visual indication may be based on an identified behavior of the driver, e.g. if the driver is looking to the right, an indication may be provided on the right side of the windscreen, indicating that the driver should look to the left.
In one embodiment of the invention, the method may further comprise logging a driver response to the provided tutorial message and determining a correlation between the desired driving behavior and the driver response.
The vehicle-integrated driver tutorial system may enable not only the determination of driving contexts, for which tutorial tasks may be performed, but also logging of the driver response after a tutorial message has been provided. The tutorial system may for example record the timing, speed, position and brake pedal pressure during navigation through a difficult crossing. Other possible parameters that may be recorded are for example the driver's reaction time, visual attention direction, use of direction indicators if needed, etc. The logged information is evaluated and compared with a desired driving behavior. The correlation between the desired driving behavior and the logged driver response may for example be used to determine or quantify the goal fulfillment or skill level of the driver for the performed tutorial task.
According to one embodiment of the invention, the method may further comprise determining a goal fulfillment based on the correlation between the desired driving behavior and the driver response. Tracking goal fulfillment can be a tool used to track the progress or skill level of a driver, which in turn can be used to provide certification or to evaluate driving lessons.
In one embodiment of the invention, the method may further comprise determining an accumulated goal fulfillment by adding a plurality of determined goal fulfillments.
The inventors have realized that a quantification and accumulation of the goal fulfillment may be useful for pedagogical development of the driver but also for further use, such as manual or automatic analysis and also adaptation of different steps in the method for the tutorial system. In the context of the present invention the quantification of the accumulated goal fulfillment may enable determination of a skill level which would correspond to the progress or proficiency of the driver, this may be quantified into at least two predetermined skill levels, where a lower level would correspond to a more novice driver and a higher level would correspond to a more skilled driver. More preferably, a finer quantification may additionally comprise an intermediate level and possibly a top performer or elite level. In other embodiments it may be more preferable to have an even finer scale, such as but not limited to a numerical scale from one to one hundred.
The skill level may be representing the general driving proficiency of the driver or the skill level in any one of the exemplified domains, categories or specific tutorial tasks. Advantageously the method could enable simultaneous tracking of separate skill levels in a plurality of tutorial tasks and based on these skill levels determine correlating skill levels in categories, that in turn may determine skill levels in domains and that in turn may determine the general skill level of the driver. It may be the case that some correlations in this chain of skill levels may be strong while other correlations are weak or even non-existing and there may also be cases where a tutorial task correlates to more than one category or domain. For example the skill level in the task of keeping a headway distance may correlate to both safe driving and efficient driving, and possibly to several categories in each domain.
The method according to various embodiments of the invention may also advantageously be used to monitor and train drivers in a commercial fleet, where a fleet manager can identify specific areas of improvement in individual drivers. The method also provides an efficient manner of teaching drivers to handle new vehicles or difficult traffic situations, reducing the need for personal tutoring. Additionally, tracking of goal fulfillment may be used as a “game” or “competition” for drivers or for groups of drivers in a fleet, thereby providing extra incentives for drivers to reach predetermined goals.
Furthermore, the operator of a fleet of commercial vehicles needs to train drivers to drive a vehicle optimally both regarding safety behaviors and regarding fuel efficiency. Improved driver behavior is strongly associated with cost reduction in the form of crash related costs, wear and tear-related costs, and fuel costs.
The driver experience is one where both the vehicle's intelligent vehicle systems and an on-line connection with an analysis center are used to give driver training tips and coaching.
Additionally, the method may comprise selecting an area of improvement to assign to the tutorial system based on the goal fulfillment or the accumulated goal fulfillment.
By determining how well the driver performs in the assigned area of improvement in relation to the desired behavior suggested by the tutorial message, the assignment of future areas of improvement can be controlled. For example, if the correlation between desired and detected behavior is low for a specific task, the tutorial system may determine that the driver should perform the task at least one more time with tutoring. On the other hand, if correlation between desired and detected behavior is high, it may be determined that the driver is capable of handling the task without the need for further tutoring. Furthermore, logged driver behavior can be provided to a database and further evaluated, either for use in offline-mode tutorials or in order to track the proficiency of a specific driver with respect to various areas of improvement. Logged driver behavior may also be used to define or update a schedule of tasks to be performed.
According to one embodiment of the invention, assigning an area of improvement may comprise manually assigning an area of improvement by the driver, remotely assigning an area of improvement by a third party or automatically assigning an area of improvement by the tutorial system. The assignment of an area of improvement may for example be performed by the driver if the driver realizes that tutoring is desirable for a particular type of driving events, or if a particular mode of driving is desired, such as a “fuel efficient mode”. As mentioned above, areas of improvement may also be set in advance or remotely by fleet managers, by driving instructors or as part of a scheduled tutorial regiment aimed at maintaining the proficiency level of particular driving tasks.
Furthermore in one embodiment of the invention the allowability of manually assigning an area of improvement by the driver may be determined based on the accumulated goal fulfillment for said driver. It may be advantageous to be able to control the possibility to manually assign an area of improvement by the driver. For example and with reference to a driver with a higher skill lever, e.g. a top performer, it may be both more efficient and result in a higher degree of driver tolerance if the driver is allowed to, and to a varying degree be responsible for, assigning areas of improvement or goals to fulfill. With reference to a driver with a lower skill level, e.g. a novice, it may be inappropriate or counterproductive to allow manual assignment of areas of improvement or goals to achieve. It may also be advantageous to vary the degree to which areas of improvement are assigned based on the accumulated goal achievement, e.g. skill level. In an example of this, a driver will experience that as the personal skill level increases, the tutorial system will increasingly encourage the driver to manually assign personal areas of improvement, and as a consequence the automatically assigned areas of improvement will decrease in number, and preferably also the areas of improvement assigned by a third party, e.g. the back office unit which could be a fleet manager or a third party service provider.
In one embodiment of the invention, the tutorial message may be selected from a plurality of predetermined tutorial messages. The tutorial message or parts of the message to be provided may be selected from a plurality of predetermined messages defined for various events. However, the message generation may also be dynamic in that the message is adapted depending on the identified operational context and driver context.
Furthermore the tutorial message may be selected based on the accumulated goal fulfillment, so that for an example referencing to skill levels, the tutorial message for one operational context may be very descriptive and contain instructions for a novice driver, while the tutorial message for an intermediate driver comprises supportive tips and the tutorial message for a top performer may comprise only a post trip review for self evaluation.
The method according to one embodiment of the invention enables the use of a driver's logged response as a future reference for the desired driving behavior. For example if a driver that achieves a better result than the desired driving behavior in terms of for example fuel efficiency or wear and tear, this may trigger the tutorial system to use this logged behavior as the future desired driving behavior.
According to another aspect of the invention there is provided an in-vehicle tutorial system for providing a tutorial message to a driver, the system comprising: a multimodal user interface, a plurality of sensors configured to determine a driving context, a processing unit configured to determine a tutorial message encouraging a desired behavior to be provided based on a driving context identified by the plurality of sensors and on an assigned area of improvement, the processing unit being further configure to provide the tutorial message to the driver through the multimodal user interface. The multimodal interface may be configured to provide audio, voice, visual, or haptic output, or combinations thereof, to the user. The visual output may for example comprise a heads-up display (HUD) arranged in the windscreen of the vehicle or similar arrangement for dynamically providing a visual message to the driver of the vehicle. Furthermore, the audio or voice output may be directional so that an alert encouraging the driver to direct his/hers attention to the right appears to come from the right side of the vehicle.
In one embodiment of the invention, the processing unit may further be configured to evaluate a desired behavior for a tutorial task in relation to the driver response to the tutorial task and to store the evaluation result in a memory storage. The memory storage may be arranged within the vehicle embodied or it may be a remote storage with which the tutorial system communicates. Furthermore, evaluation may be performed immediately after a task has been completed, or tasks and responses can be stored for later evaluation by an external system.
Further effects and features of this aspect of the present invention are largely analogous to those described above in connection with the previous aspect of the invention.
According to another aspect of the invention there is provided a computer readable medium embodying a computer program product for providing a tutorial message to a driver in a vehicle using an in-vehicle tutorial system comprising a multimodal user interface, the computer program product comprising code configured to, when executed by a processor:
assign a tutorial task to the tutorial system, identify a driving context for which the tutorial task is to be performed, based on the tutorial task and the driving context, select a tutorial message encouraging a desired driving behavior, and provide the selected tutorial message to the driver using the multimodal user interface.
The processing unit may preferably be provided in a vehicle control unit, a computer, server or similarly, and the computer readable medium may be one of a removable nonvolatile random access memory, a hard disk drive, a floppy disk, a CD-ROM, a DVD-ROM, a USB memory, an SD memory card, or a similar computer readable medium known in the art (present and future). The present invention may be implemented using a combination of software and hardware elements.
Further features of, and advantages with, the present invention will become apparent when studying the appended claims and the following description. The skilled addressee realize that different features of the present invention may be combined to create embodiments other than those described in the following, without departing from the scope of the present invention.